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Abstract 

It is well known that the B-spline filter can yield a very 
accurate algorithm for smoothing. In this paper, it is shown 
that a cubic B-spline filter can be used to improve the non- 
linear image enhancement method. In the non-linear image 
enhancement, a higher-frequency component can be 
predicted to solve the blurred problem of an enlarged image. 
This paper also presents a new three-dimensional (3-D) 
down-scaling scheme to subsample video data for video 
coding. Furthermore, a novel non-linear image-enhancement 
compensation algorithm with cubic B-spline filter is 
presented to improve the prediction of higher-frequency 
component and accordingly the efficiency of the 3-D down- 
scaling video coding. Finally, a computer simulation shows 
that the proposed method yields a better quality of the 
decoded image than other non-linear enhancement methods. 

Keywords 

Cubic B-spline Filter; Non-linear Image Enhancement; Video 
Coding 

Introduction 

With the growing interest of digital image processing, 
the applications in this domain such as digital high 
definition television (HDTV), Internet protocol 
television (IPTV), and video-phone, an integral part of 
our life are related to a scale image enlargement 
technique. Typically, however, the image enlargement 
causes a blurred image because there is no power in 
the high-frequency component of enlarged image [14]. 
To improve the quality of such a blurred image, the 
image enhancement is an indispensable post- 
processing method. 

Image Enhancement is a very important topic in the 
researches. The principle of image enhancement is to 
process an image so that the result is more suitable 
than the original image in many applications. A 
typical image enhancement is achieved througth the 
high-pass filter followed by the post-processing in 



order to make the image suitable. In other words, this 
method uses a typical principle behind un-sharp 
masking and high-boost filtering [14]. 

Non-linear image enhancement [1, 2, 13] is similar to 
the typical image enhancement except that the high- 
pass filter is replaced by non-linear operations. This 
enhancement method uses the Gaussian-pyramid [1] 
or filter subtract and decimate (FSD)-pyramid [2] 
representation of an image to extract the high- 
frequency component from input (blurred) image. 
That is, a high-frequency component L-i can be 
obtained from a blurred image by a nonlinear filter as 
shown in Fig.l. A new output image is generated next 
as the sum of the given input image and high- 
frequency component L-i. The major non-linear step 
involves clipping and scaling the extracted 
components. 



Input Image 
■ 




Nonlinear Filter 



Output Image 




FIG. 1 THE NON-LINEAR IMAGE ENHANCEMENT 

The cubic B-spline filters have been extensively used 
in image processing [3, 6, 7, 8, 10]. In [11] part, which 
is related to an improved non-linear image 
enhancement method for video coding, is proposed. 
However, the detailed derivation of both non-linear 
image enhancement with the cubic B-spline filter and 
the 3-D down-scaling video coding has not been 
presented. In addition, this paper gives more detailed 
descriptions on the non-linear image enhancement 
compensation with the cubic B-spline filter and the 3- 
D down-scaling video coding. Furthermore, the novel 
non-linear image enhancement compensation is used 
along with video coding to improve the quality of 
decoded image. Finally, some experimental results 
show that the proposed method obtains a better 
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subjective quality and objective PSNR performance 
than other non-linear image enhancement methods. 

The rest of this paper is organized as follows. Section 2 
describes the cubic B-spline filter. The non-linear 
image enhancement is proposed in Section 3. Section 4 
shows the 3-D down-scaling scheme for video coding. 
In Section 5, the video coding using non-linear image 
enhancement compensation is described. The 
computer simulation is illustrated in Section 6. The last 
section shows the conclusions of this paper. 

Cubic B-Spline Filter 

Splines are piecewise polynomials with pieces that are 
smoothly connected together. The B-splines (where the 
B may stand for basis or basic) are the basic building 
blocks for splines [8]. In addition, it is shown in [3, 6, 7, 
10] that the B-spline is a very good low-pass filter for 
image representation. 

Let £ :£o <€\<~-<€ n < % n +\ be a partition of the 
interval [ <f , % n+l ] on a real axis. The spline basis 
function of degree n on £ in [3] is the following 
piecewise polynomial: 

-(.+PZ tf - ftr ,??- ft> ■«» -cm- (1) 



where 



and 



n+l 

co({ k ) = H({ k -{j) 

7=0 
j*k 



L ,for£<4j 
is a unit step function. 

From (1), the cubic B-spline function is given by 

s 3 (#-4) 

=[(#-&_ 2 ) 3 tf(£-&_ 2 ) 

-4(^-4_ 1 ) 3 t/(^-4-i) 
+ 6(#-^) 3 f/(^-4) 

-4(|-4 +1 ) 3 t/(^-| t+1 ) 

+ (|-^ +2 ) 3 C/(|-^ +2 )]/6A 4 , 



(2) 



where A = % k -% k _ l . 

An interpolation function can be expressed in the 
following form 



N 



(3) 



k=\ 



where c k is the coefficients to be determined from the 
input data, S k {^-^ k )is the spline basis function, and 
N is the number of given data points. 




\ i i \x i i i i r 
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FIG. 2 /(£) INTERPOLATED BY CUBIC B-SPLINE 

From (l)-(3) and illustrated in Fig.2., the cubic B-spline 
interpolation function in one-dimension (1-D) can be 
obtained as 

m = {c k -M-^f +6(^-4-1) 

-4(?-4) 3 ]+q[(?-4- 2 ) 3 -4(^-4-i) 3 
+6{^-4) 3 ]+c, +1 [(^-4_ 1 ) 3 -4(^-4) 3 ] {4) 

+c, +2 [(^-4) 3 ]}/6A 4 , 

where ^_ 1? c^,c^ +1 and c^ +2 are the coefficients to be 
determined from the input data. 

Let £ = g k + xA , where < x < 1 . The cubic B-spline 
interpolation function in (4) can be written as 

= {c k _ x [(3 + x) 3 - 4(2 + x) 3 + 6(1 + x) 3 - 4x 3 ] 
+ cj(2 + x) 3 -4(l + x) 3 +6x 3 ] 



+ c, +1 [(l + x) 3 -4x 3 ] 
+ c k+2 x 3 }/6A 
= { x3 ( c k+2 ~ 3c k+l + 3c k - c k _ x ) 
+ x 2 (3c k+ i -6c k +3c k _i) 
+ x(3c k+l -3c k _ l ) 
+ (c k+l +4c k +c k _ l )}/6A. 



(5) 
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Then (5) can be used to find the interpolation at any 
point among sampled points. In particular, at the node 
point % = % k , i.e., x = , (5) becomes 



m) = (c k+l + 4c t +c t _,)/6A. 



(6) 



In other words, the 1-D filter of cubic B-spline function 
in (6) is [1, 4, 1] / 6. 

Based on the definition of the two-dimensional (2-D) 
interpolation function [10], the cubic B-spline 
interpolation function can be extended from the 1-D 
interpolation function to the 2-D interpolation function. 

Let 77) be the 2-D cubic B-spline interpolation 
function at the point ( £ = g k + xA , rj = rj l + yA), where 
< x < 1 and < y < 1 . By (5), one obtains 



(7) 



/(*, J) = ifi-i «[(3 + yf - 4(2 + y) 3 + 6(1 + y) 3 
-4y 3 +/ ; W[(2 + y) 3 -4(l + j) 3 +6y 3 ] 

+ /,+, (*)[(! + y? - V ] + f l+2 (x)y 3 }/6A 

In particular, at the node point ( g k , t] l ), i.e., x = and 
y = , (7) gives 

f(€k>m) = i( c k-i,i-\ + 4 ^,/-i +^ + u-i) 

+ 4(^_ u +4c^+c, +u ) (8) 

for all lfc = l,2, — ,£ and / = 1,2, — ,L. 

In other words, the 2-D filter of cubic B-spline function 
in (8) is [1, 4, 1; 4, 16, 4; 1, 4, 1] / 36. 

The Proposed Enhancement Scheme 

In this paper, the proposed enhancement algothrim 
applies the philosophy of the cubic B-spline filter to 
improve the non-linear image enhancement method [2] 
which is shown in Fig.3. 










Non-linear operator 










FIG. 3 THE PROPOSED NON-LINEAR IMAGE ENHANCEMENT 
METHOD 



The low-frequency image I x is obtained from the 
input-blurred image I using the cubic B-spline filter, 
and the high-frequency image K , called the residual 
image, is obtained by subtracting the low-frequency 
image I { from the input-blurred image 7 , i.e., 



k o= i q- i i- 



(9) 



By [2], the enhanced image I_ x is generated as the sum 
of the input-blurred image I and the predicted 
higher-frequency image K_ x ; that is, 



I -1 =I +K- U 



(10) 



where K_ x = NL(K ) is a non-linear operator of K , 
which includes both scaling and clipping steps, 
defined as follows: 



NL(K ) = sxCUp(K ) 



0) 9 



(11) 



where the scaling constant s ranges in between 1 and 
10 and Clip(x) is given by 



Clip(x) 



7. ifx>T 

x, if-T<x<T 



(12) 



-T, ifx<-T 

where x is the pixel of the high-frequency image K , 

T = c * K o ma „ (13) 

where ^ 0m ax ^ s tne maximum pixel of the high- 
frequency image K and the clipping constant c ranges 
in between and 1. 

After a non-linear operator, the higher-frequency 
image K_ x can be utilized to enhance the input-blurred 
image / . 

Then, using the description in [2], the values of c and s 
parameters are estimated as follows. Let the standard 
deviation <r of input image I be 0.9, the standard 
deviation a A of enhanced image I_ x be 0.45, and the 
low-pass (cubic B-spline) filter LF is normal 
distribution, whose standard deviation <r LF be 1. 
Through the low-pass filter LF, the low-frequency 
image I x is acquired. So that the standard deviation <j x 

of I x is 1.345 (o- 1 2 =t7 2 +o- LF 2 =1.81). The residual 
image K Q in (9) is obtained by subtracting I x from I . 
Through the normalized Gaussian filter, the 
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ErrorFunction is generated. That is, 

K = Erf(x/(a ))-Erf(x/(a x )) . (14) 
Therefore, the maximum value can be obtained as 

£r/'(xmax/(cr )) - Erf'(xmzK/(a x )) = . (15) 



or 



and 



'(To) 



0/xmax 



1 /imax 



= 0, 



(16) 



xmax^^r^log^/cro)/^/^) 2 -l/^) 2 )) (17) 

It is followed from [2] that the solution of (14) and (17) 
can be confirmed. Thus, one parameter combination of 
c = 0.45, s = 3 from the theoretical evaluation and the 
other parameter combination of c = 0.4, s = 5 from the 
estimation analysis are proposed for the non-linear 

A 3-D Down-scaling for Video Coding 

In order to obtain a low bit-rate video, a new type of 3- 
D down-scaling scheme [11] is presented for video 
coding. In general, the process of decreasing the data 
rate is called decimation while the process of 
increasing data samples is called interpolation [3]. This 
3-D down-scaling scheme applies a 3-D decimation 
with a compression ratio of 8 to 1 as the pre- 
processing step of the encoder. As a consequence, a 3- 
D interpolation with a ratio of 1 to 8 is used for the 
post-processing step of the decoder. 

A 3-D Linear Decimated Scheme 

Let t x ,t 2 aR d h be the integer indices and n x ,n 2 and 
n 3 are also integers. The 3-D decimated scheme [12] 
takes an video x(t x ,t 2 ,t 3 ) as an input and produces an 
output of y(t x , t 2 ,t 3 ) by a factor of 2 in each dimension 
as follows: 



Y(t x ,t 2 ,t 3 ) = 

l l l 

ay g(Z Z Z mh+i x ,2t 2 +i 2 ,2t 3 +i 3 )) 

i,=0 / 2 =o z 3 =0 

for 0< t { <n { -1, i = 1,2,3. 



(18) 



where avg(?) is returns the average (arithmetic mean) 
of a set of numeric values. Fig.4. shows the down- 
sampling of pre-processing stage using the 3-D linear 
method. In this figure, a simplified example with two 
adjacent 4x4 frames, that is, frame N and frame N+l, is 
illustrated. Firstly, two adjacent 4x4 frames are 



divided into four separate 2x2 groups as depicted in 
Fig.4(a). Then, each 2x2 group is calculated by (18) to 
obtain a corresponding decimated data of Y(t x ,t 2 ,t 3 ) as 
shown in Fig.4(b). 

A 3-D Linear Interpolated Scheme 

It is followed from [9] that a symmetric extension 
scheme is used to solve the boundary condition 
problem to compute the 3-D interpolation at both 
boundaries of the image. To illustrate this, we consider 
the 1-D case, for example, in Fig. 5, if Xk for k = 
0,1,2, ... ,7 is original data, then X-k= Xk and X7+k= X7-k 
are extended from the left and right boundaries of Xk, 
respectively. 




fb) 



FIG. 4 A 3-D LINEAR DECIMATED SCHEME 
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FIG. 5 SYMMETRIC EXTENSION OF IMAGE DATA 
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FIG. 6 A 3-D LINEAR INTERPOLATED SCHEME 
Using the decimated video Y(t x ,t 2 ,t 3 ) obtained from 
(18), the 3-D reconstructed video [12] can be calculated 
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by a linear interpolation shown in Fig.6. and given by 



x(t v t 2 , t 3 y. 
i i i 



- (19) 



£ X Z F (*i> k 2> hWih -2*i, h -2* 2 , r 3 -2* 3 ; 

Jk 1= 0Jfe 2 =0Jfe 3 =0 

for 0< ^ <3, i = 1,2,3. 

where R(t l -2k l ,t 2 -2k 2 ,t 3 -2k 3 ) is the 3-D linear 
functions defined as 

R(t l9 t 29 t 3 ) = R(t l )R(t 2 )R(t 3 ) 
and R(t) is the 1-D linear function given by 

i-kl/2 



(20) 



*(*) = 



(21) 



'It 2 

, otherwise 

That is, in the 3-D linear interpolated scheme, using 
the reconstructed values at the decimated video points 
Y(t l9 t 29 t 3 ), the reconstructed video points X(t v t 2 , t 3 ) 
between the decimated video points are obtained by 
the use of (19). A simplified example with the 2x2 



decimated video points of Y(t l9 t 2 ,t 3 ) that are 
interpolated is depicted in Fig. 6(b) and in Fig.6(a) a 
mirrored method based on the above symmetric 
extension scheme is used to solve the boundary 
condition problems of computing the X(t v t 2 , t 3 ) in 
(19). Finally, the two 4x4 reconstructed video frames 
are obtained as shown in Fig.6. 

Computation Complexity 

In order to illustrate the computation complexity of 
the proposed 3-D down-scaling scheme, the number of 
multiplication and addition/subtraction/shift is 
estimated in Table 1. In this table, the operations are 
categorized into two groups: one is multiplication 
operation and the other is addition, subtraction, and 
shift operation. Obviously, the estimated operation 
numbers of the proposed 3-D down-scaling scheme in 
both dicimation and interpolation are very compact 
and thus the proposed 3-D down-scaling scheme can 
be realized quite easily in real time. 



TABLE 1. NUMBER OF OPERATIONS FOR 3-D DOWN-SCALING SCHEME. 



Function 


Unit 


Multiplication 


Addition, 
Subtraction, 
Shift 


Estimation Basis 


3-D Linear 
Decimated 
Scheme 


(4X4)X2 
blocks 
-2X2 block 





36 


3-D decimation of spatial (vertical, horizontal) and temporal 
directions are performed independently, 
vertical direction: (4 X 2) X 2=16 addition 
horizontal direction: (2 X 2) X 2=8 addition 
temporal direction: 2 X 2=4 addition 
rounding: 2 X 2=4 addition 
divided by 8: 2 X 2=4 3-bit shift 


3-D Linear 
Interpolated 
Scheme 


2X2 block- 
(4X4)X2 
blocks 





120 


3-D interpolation of spatial (vertical, horizontal) and temporal 
directions are performed independently, 
vertical direction: 4 X 2=8 addition 
horizontal direction: 4X4=16 addition 
temporal direction: (4 X 4) X 2=32 addition 
rounding: (4 X 4) X 2=32 addition 
divided by 8: (4 X 4) X 2=32 3-bit shift 



Video Coding Using Non-linear Image 
Enhancement Compensation 

The 3-D down-scaling for video coding provides a 
better performance at a lower bit-rate transmission, 
however, this method causes the blurred problem of 
decoded image. In this section, the proposed non- 
linear image enhancement compensation with the 
cubic B-spline filter is used to improve the decoded 
quality of the 3-D down-scaling for video coding 
shown in Fig. 7. In this figure, this algorithm applies 
the 3-D linear decimation (3D-LI^) as the encoder, and 
the 3-D linear interpolation (3D-LIf) as the decoder for 
video coding. As a consequence, the proposed non- 



linear image enhancement compensation with cubic B- 
spline filter is used for the post-processing step of this 
decoder. 

For this algorithm, an original video (image) sequence 
in the RGB color space is converted into another 
preliminary sequence in YUV [4, 5] color space prior to 
the 3D-LIJ, processing. The size of the original RGB 
video sequence in 352x288 resolution is assumed to be 
352x288x3x30 bytes for a period N = 30 frames. After 
color-space conversion, one set of 352x288x30 bytes is 
used for Y, and two sets of 176x144x30 bytes are used 
for the U and V video sequences. In the proposed 3D- 
LIJ, procedure, the input video sequence is a Y video 
sequence of size 352x288x30 bytes, and the output 
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video sequence is a decimated video sequence of size 
176x144x15 bytes. For the U and V video sequences, 
the input video sequence has 176x144x30 bytes so that 
the output video sequence to be decimated is 88x72x15 
bytes. At the end of the encoder, the three separate Y, 
U, and V decimated video sequences are combined 
into one YUV decimated video sequence. 

In addition, in the decoder, there are two processes 
used that are reversed in the encoding steps. In the 
first step, the YUV received (decimated) video 
sequence is separated into three separate Y, U, and V 
decimated video sequences. Then, the proposed 3D- 
Llf process uses the linear interpolation to reconstruct 



the video sequences. After this interpolation, the size 
of the Y video sequence is therefore converted from 
176x144x15 bytes to 352x288x30 bytes, and the U and 
V video sequences are increased from 88x72x15 bytes 
to 176x144x30 bytes. In the second step, the proposed 
image enhancement compensation method, described 
in Section 3, is used for the Y video sequence only, and 
it is not used for the U and V video sequences. 
Furthermore, the three Y, U, and V video sequences 
are combined again into one YUV format. Finally, it is 
followed from [4, 5] that this YUV video sequence is 
converted into the reconstructed RGB video sequence 
with size 352x288x3x30 bytes. 



Input Image 
Sequence 



Encoded Image 
Sequence 



Received Image 
Sequence 




Decoded Image 
Sequence Buffer 



Current Image 
Frame 



High-frequency Image 
Buffer 



Image Enhancement Compensation 



FIG. 7 VIDEO CODING USING NON-LINEAR IMAGE ENHANCEMENT COMPENSATION 



Computer Simulation 

Let X(i,j) and X(i,j) be the original (input) and 
reconstructed images, respectively, and let i,j for 
< i < M - 1 and < j < N - 1 be the index numbers 
that determine the vertical and horizontal positions of 
pixels in the images. The mean square error (MSE) 
between X(i,j) and X(i 9 j) is given by 



MSE= 



M-W-l 



X(i,j)-X(i,j) 



/(MxAT). (22) 



II- 

v f=0y=0 

Thus the peak signal-to-noise ratio (PSNR) between 
X (i, j) and X (i, j) is defined by 



PSNR(dB) = I0\og lo (b 2 /MSE) , 



(23) 



where b is the largest value of the image signal 



(typically 255 for 8-bits of gray level). 

In this section, some experimental results of three gray 
images (Aerial, Baboon, Barbara) in 512x512 resolution, 
three color images (Lena, Peppers, Sailboat) in 512x512 
resolution and three video sequences (Mother, Stefan, 
Table) in 352x288 resolution are presented by 
computer simulations. First of all, these gray and color 
images are blurred to low-resolution images, 
repectively, then the proposed method and the 
methods in [1] and [2] are used to enhance these low- 
resolution images, in addition, the PSNR of these 
experimental results are compared in Table 2 and 
Table 3. Furthermore, the 3-D down-scaling for video 
coding is used along with the proposed method and 
the methods in [1] and [2] to enhance the 
reconstructed images of the above three video 
sequences, finally, the PSNR of these experimental 
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results are also compared in Table 4. Obviously, in 
Tables 2, 3 and 4, the quality of reconstructed images 
using the proposed method is better than the methods 
in [1] and [2]. 

Note that in clipping and scaling parameters, s=5, 
c=0.4 are selected for both gray and color images, but 
for 3-D down-scaling video coding s=3, c=0.45 are 
chosed, because these parameters get better PSNR 
results. 

TABLE 2. PSNR (DB) OF GRAY ENHANCED IMAGE OF SIZE 512x512 FOR 
GAUSSIAN[1], FSD[2], AND PROPOSED METHODS. 



Image 
Name 


Blurred 
Image 


Gaussian 
method[l] 


FSD method [2] 


Proposed 
method 


0.04xG 0max 


s=3,c=0.45 


s=5,c=0.4 


s=5,c=0.4 


Aerial 


25.57 


25.81 


27.94 


25.12 


28.46 


Baboon 


21.88 


22.20 


23.49 


22.59 


23.50 


Barbara 


24.68 


24.54 


25.58 


24.87 


25.72 






TABLE 3. PSNR (DB) OF COLOR ENHANCED IMAGE (Y) OF SIZE 512x512 
FOR GAUSSIAN[1], FSD[2], AND PROPOSED METHODS. 



Image 
Name 


Blurred 
Image 


Gaussian 
method[l] 


FSD method [2] 


Proposed 
method 


0-04xG 0max 


s=3,c=0.45 


s=5,c=0.4 


s=5,c=0.4 


Lena 


31.97 


30.79 


34.36 


30.79 


34.81 


Peppers 


29.80 


28.97 


31.18 


29.28 


31.44 


Sailboat 


27.62 


27.43 


29.66 


26.97 


29.93 



TABLE 4. PSNR (DB) OF ENHANCED VIDEO SEQUENCE (Y) OF SIZE 352x 
288 BY 3-D DOWN-SCALING FOR GAUSSIAN [1], FSD[2], AND PROPOSED 
METHODS. 



Sequence 
Name 


3D-LI I 

/3D-LI f 


Gaussian 
method[l] 


FSD method [2] 


Proposed 
method 


0.04xG 0max 


s=3,c=0.45 


s=5,c=0.4 


s=3,c=0.45 


Mother 


33.97 


32.74 


33.74 


29.93 


35.19 


Stefan 


21.43 


21.60 


21.93 


20.64 


22.04 


Table 


24.07 


24.26 


24.54 


22.92 


24.94 





./- 

r 




(a) Original image 




B 



Q if 



(b) Blurred image 



(c) Gaussian method(o.04xG 0max )[l] 

w 





- .n 




(d) FSD method(s=3,c=0.45)[2] (e) FSD method(s=5,c=0.4)[2] (f) Proposed method(s=5,c=0.4) 

FIG.8 COMPARISON OF SUBJECTIVE QUALITY OF ENHANCED GRAY AERIAL IMAGE. 



Finally, in Figs. 8, 9 and 10, clearly, the enhanced image 
by means of the proposed method indicates a better 
subjective quality than that by other enhanced 
methods. Though the FSD method is the clearest 



algorithm, it causes the ring effect for the enhanced 
image because the higher-frequency component is 
enhanced too much, and the Gaussian method is 
inferior in performance to the proposed method and 
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the FSD method. 




(a) Original image (b) Blurred image (c) Gaussian method( o.04 x G 0max )[1] 




(d) FSD method(s=3,c=0.45)[2] (e) FSD method(s=5,c=0.4)[2] (f) Proposed method(s=5,c=0.4) 

FIG.9 COMPARISON OF SUBJECTIVE QUALITY OF ENHANCED COLOR SAILBOAT IMAGE (ZOOM IN). 




(d) FSD method(s=3,c=0.45)[2] (e) FSD method(s=5,c=0.4)[2] (f) Proposed method(s=3,c=0.45) 

FIG.10 COMPARISON OF SUBJECTIVE QUALITY OF ENHANCED COLOR MOTHER IMAGE (VIDEO SEQUENCE). 
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Conclusions 

In this paper, the proposed algorithm uses the cubic B- 
spline filter and the non-linear image enhancement 
compensation to improve the blurred image and video 
sequence. Such a new method is not only efficient but 
also superior in performance to the other image 
enhancement methods. Experimental results also 
show that the proposed method yields a better 
subjective quality and objective PSNR than other non- 
linear image enhancement methods for the 
reconstructed image. Furthermore, the proposed 
algorithm can be realized quite easily in real time and 
also can be used with the scalable video coding [4] for 
the digital broadcast TV/HDTV, IPTV, and video- 
phone applications. 
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